[论文精读] OasisKV: Scaling In-Decode KV Cache
A memory-centric LLM inference system that decouples full KV-cache storage from HBM, using lookahead tokens from speculative decoding to prefetch only the most relevant KV blocks, achieving 1.69×-2.3× throughput gains within 0.7 points of full-attention accuracy.